The Measurement Manager: Modular End-to-End Measurement Services

Network measurement is used to improve the precision, efficiency, and fairness for a variety of Internet protocols and applications, ranging from transport protocols to overlay networks. Measurement is typically performed in one of two ways: (1) passively, by using existing data traffic to perform measurements, typified by transport protocols like TCP and passive measurement infrastructures or (2) actively, by injecting special probe packets into the network to measure network properties of end-to-end paths, typified by overlay networks that use ping, traceroute and standalone active measurement tools. Neither approach is entirely satisfactory. The passive approach is limited by the shape of application traffic and tends to be tightly coupled with the protocol implementation, hurting modularity. The active approach is faster, more accurate and more modular but imposes a significantly higher overhead, especially at large scales. As a remedy to these problems, this research proposal presents the Measurement Manager architecture, a modular approach to performing end-to-end network measurement that includes the best features of the passive and active measurement approaches. Like the active approach, the Measurement Manager allows measurement algorithms to be specified separately from applications, exporting their results as a service to both transport protocols and applications. Like the passive approach, however, the Measurement Manager implements active algorithms in a way that allows probes to be transparently piggybacked on top of existing transport traffic, namely TCP. The piggybacking happens on-demand according to various tuning knobs that effectively trade off accuracy and overhead. We propose to evaluate the Measurement Manager architecture in terms of its utility, accuracy, and performance by applying it to several real-world applications and transport protocols and performing experiments that illuminate tradeoffs in the measurement process.